docs(databricks): Lakeflow pipelines, a build whose expectations are the contract's DQ rules (forge-cli#872) - #213

What

Documents forge-cli #872: a build with engine: lakeflow (the 0.7.7 preview schema) runs as a Lakeflow Declarative Pipeline that owns its table, and the contract's dq.rules become its expectations. It shipped in forge-cli 0.30.0.

docs/providers/databricks.md gets a "Lakeflow pipelines" section, before "Landing a DuckDB acquisition":

• opting in: the build, the settings it takes, and the refusal on a 0.7.6 contract;
• what fluid apply provisions, with the pipeline source forge-cli generates for the worked example;
• rules as expectations: each rule type and severity, thresholds, rule ids, and references to another product;
• running it from fluid apply: each mode, a live amend-and-build capture, the per-rule marks, a failed update, how long it waits, dry-run, and the tag sync on the pipeline's table;
• what is refused next to a pipeline's table, and what is not set up on it yet (Unity Catalog keys, data quality monitoring, a metric view, the Genie space), with the fluid validate warning;
• what a lakeflow build refuses, and where (validate, generate iac and apply, not plan);
• fluid verify on a pipeline's table;
• a two-product worked example, the same contracts as the live runs.

Plus one line each in that page's overview, prerequisites, provisions table, validate list, verify pointer, Limitations and "How it is tested", and one in the Metric views intro (a pipeline's table gets no view yet). The dbt run-now section's quote of apply_databricks_sdk_missing now has the text #872 gave it ("a dbt or lakeflow build ...").

Elsewhere, one marked sentence each in cli/apply.md (the build phase, the SDK error row), concepts/builds-exposes-bindings.md, providers/README.md, advanced/operating-in-ci.md and advanced/environment-variables.md (FLUID_DATABRICKS_RUN_TIMEOUT_SECONDS also caps a pipeline update).

Status

0.30.0 contains #872 (d2ddd7ce), and 0.30.1 changes nothing this documents, so every marker is the release form, *(since 0.30.0, [forge-cli #872](<url>))*.

The draft was written against forge-cli 84da9dde (the commit v0.30.0 was cut from), then rebased onto main after #211 and #212. The one conflict was the end of "How it is tested", where #211 added two paragraphs. Both stay, followed by this PR's not-proven line. The second commit links the new Metric views section from the Lakeflow list, and says in that section that a pipeline's table gets no view.

Verification

• Both worked-example contracts validate on main and are the live runs' contracts. The pipeline SQL block is byte-identical to fluid generate iac output.
• The live captures come from the implementing session's Free Edition runs (2026-11-12 and 2026-11-13), with the host and ids replaced. The 2026-11-12 failure predates the record cut and the warn mark, so the page reprints it through the released _report_quality and _errors code, and says so.
• The quoted fluid validate and fluid generate iac output (the 0.7.6 refusal, the SLO and semantics warnings, the cron refusal) was re-run on 0.30.1 (9980ac6d) and is unchanged. #876's notes channel only moved the dbt-comments message.
• A verifier agent (sonnet, high effort, in its own CLI worktree) checked the claims against the code and the CLI. It found one wrong claim (the release status: 0.30.0 was tagged mid-draft) and 13 imprecise ones, all fixed. They covered view names, thresholds, which comment is refused as what, when tags are set, the continuous-pipeline caveats, and lists written as complete.
• scripts/check_cli_docs.py (full, --version-only, and --markers-only against forge-cli main's CHANGELOG, newest release 0.30.1), scripts/check_providers.py and scripts/gen_contract_reference.py --check, with the pinned CLI 0.29.0 on Python 3.11: all pass.
• npm ci && npm run docs:build and node scripts/check-dist-links.mjs: pass. All 22 #anchor links the change adds resolve to heading ids in the built HTML. lychee offline over the source, with CI's route mirror: 0 errors, canaries 0 and 2.
• The 107 Lakeflow unit tests pass at 84da9dde.
• Not proven live, and the page says so: replace-and-build, a continuous pipeline, a table that exists outside its pipeline, a grant to a real group, a steward's tag, a tag list longer than one page, a failing critical rule with today's output, and the Data quality tab itself (the counts are the event log's).

Left alone

• docs/upgrading.md: the engine is opt-in (a 0.7.7 contract with engine: lakeflow), so nothing changes for an existing contract, and #872's CHANGELOG lines are all under Added.
• The markers for #861 (branch form), #864 and #871 still say "not yet in a release", although 0.30.0 contains all three. That is the 0.30.x pin bump's job. Several of them share lines with this PR, so the bump is easier after it.
• Genie (#863, #869, #870) and Unity Catalog keys (#860) have no section on the page yet, so the Lakeflow list links their PRs.
• fluid generate iac emits a module for a 0.7.6 contract with engine: lakeflow, because it does not schema-validate; fluid validate and fluid apply refuse it. That is a forge-cli follow-up, not a docs one.
• The CLI pin stays at 0.29.0.